Generalized Cross-Entropy Methods
نویسندگان
چکیده
The cross-entropy and minimum cross-entropy methods are well-known Monte Carlo simulation techniques for rare-event probability estimation and optimization. In this paper we investigate how these methods can be extended to provide a general non-parametric cross-entropy framework based on φ-divergence distance measures. We show how the χ distance in particular yields a viable alternative to Kullback-Leibler distance. The theory is illustrated with various examples from density estimation, rareevent simulation and continuous multi-extremal optimization.
منابع مشابه
New Generalized Parametric Measures of Entropy and Cross Entropy
The measure of entropy introduced by Shannon [12] is the key concept in the literature of information theory and has found tremendous applications in different disciplines of science and technology. The various researchers have generalized this entropy with different approaches. The object of the present manuscript is to develop a generalized measure of entropy by using the property of concavit...
متن کاملGeneralized Cross-entropy Methods with Applications to Rare-event Simulation and Optimization
The cross-entropy and minimum cross-entropy methods are well-known Monte Carlo simulation techniques for rare-event probability estimation and optimization. In this paper, we investigate how these methods can be extended to provide a general non-parametric cross-entropy framework based on 1-divergence distance measures. We show how the 2 2 distance, in particular, yields a viable alternative to...
متن کاملCombinatorial Information Theory: I. Philosophical Basis of Cross-Entropy and Entropy
This study critically analyses the information-theoretic, axiomatic and combinatorial philosophical bases of the entropy and cross-entropy concepts. The combinatorial basis is shown to be the most fundamental (most primitive) of these three bases, since it gives (i) a derivation for the Kullback-Leibler cross-entropy and Shannon entropy functions, as simplified forms of the multinomial distribu...
متن کاملNonlinear Image Processing Based on Optimization of Generalized Information Methods
A range of nonlinear image reconstruction procedures based on extremizing the generalized Shannon entropy, KullbackLeibler cross-entropy and Renyi information measures and proposed by the author in early papers is presented. The “generalization” assumes search for the solution over the space of real bipolar or complex functions. Such an approach allows, first, to reconstruct signals of any type...
متن کاملA pr 2 00 7 Combinatorial Information Theory : I . Philosophical Basis of Cross - Entropy and Entropy
This study critically analyses the information-theoretic, axiomatic and combinatorial philosophical bases of the entropy and cross-entropy concepts. The combinatorial basis is shown to be the most fundamental (most primitive) of these three bases, since it gives (i) a derivation for the Kullback-Leibler cross-entropy and Shannon entropy functions, as simplified forms of the multinomial distribu...
متن کامل